Variance Explained: Why Size Does Not (Always) Matter

I examine the role of explaining variance in the construction of explanatory theory. Explaining variance can be an insufficient basis for evaluating a theory (Lieberson, 1985). Starting with this insight, I suggest that models that provide explanations of variance do not necessarily provide good explanations of causal mechanisms. I then explore the utility of process models and theories (Mohr, 1982) relative to variance theories. I clarify the role of stochastic processes in such model building and discuss the implications of such processes for evaluating explanatory ‘adequacy’. Under some conditions, explaining variance may be neither a necessary nor a sufficient condition for good explanatory theory. I then identify some implications of this argument for developing and analyzing explanatory theory. These arguments are applied to two examples: (1) meta-analysis and (2) the disposition versus situation debate (a variant on the nature vs. nurture argument) to illustrate the implications of this process theory point of view. The Variance Explained Criterion The Pursuit of Variance Organizational and social researchers frequently try to build variance theories (Mohr, 1982). The goal is to develop a theoretical structure that is consistent with (explains variance in) the data (Pfeffer, 1982) while being parsimonious, interesting and conceptually sound. Given the applied underpinnings of organizational research, we are interested in developing theories that maximize this consistency; explain as much of the variance as possible1. A common assumption in our field is that the amount of variance explained is an indicator of the quality or explanatory power of a theory. In reading papers, we consider goodness of fit statistics like R or η and use them to evaluate explanatory adequacy. When effect size statistics are lacking, statisticians develop them in order to meet the demand for them (e.g., Bentler & Bonett, 1980). Effect size statistics are an essential requirement in any research report (Rosenthal & Rosnow, 1991), allowing readers to assess more than the significance of the effect, but the strength of the effect itself. However, we often subtly move from assessing strength of effects in a study to using effect sizes to assess the underlying theory, indexing the explanatory power of a theory by strength of effects. In some cases, this becomes a sort of competition or race between theories for which theory or model explains the most variance, and hence is declared the winner in I use explanatory theory or explanation with reference to theory in the sense of providing a causal account (Salmon, 1984). The usage ‘explain variance’ and its variants are a different usage not invoking this sense of a causal accounting. Rather, explaining variance is a common usage meaning a statistical association between an independent variable set and a dependent variable set. The slippage between and confusion of these two senses, a causal account or explanation and a statistical association, explaining variance, is what I am concerned with here.